Added support for the "think" for Ollama #3386

sunyuhan1998 · 2025-05-30T03:28:17Z

Fixes #3383
As mentioned in issue: #3383 , Ollama added support for "think" in its latest 0.9.0 version:
https://github.com/ollama/ollama/releases
https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion.

This PR implements support for that attribute and includes the following key changes:

Added the think field to Ollama's ChatRequest
Added the thinking field to Ollama's Message
Added the think property to OllamaOptions, allowing users to specify whether to enable or disable thinking

Actually, there is currently another issue:

As stated in Ollama's API documentation here, during requests to Ollama, the message field supports sending the model's own reasoning (thoughts) back to it. However, AssistantMessage does not currently support transmitting this field, which means the model will not be aware of its previous thoughts.

Therefore, perhaps we need to add a specialized Message implementation for Ollama, such as OllamaAssistantMessage. I'm not sure whether this would be considered a significant change.

1. Added the `think` field to Ollama's `ChatRequest` 2. Added the `thinking` field to Ollama's `Message` 3. Added the `think` property to `OllamaOptions`, allowing users to specify whether to enable or disable thinking Signed-off-by: Sun Yuhan <[email protected]>

Signed-off-by: Sun Yuhan <[email protected]>

…fault behavior, thereby ensuring compatibility with older versions of Ollama calls. Signed-off-by: Sun Yuhan <[email protected]>

sunyuhan1998 · 2025-05-30T03:58:06Z

@tzolov @ilayaperumalg @markpollack Could you please help review this PR? Thank you.

…tainer image version of ollama to 0.9.0 Signed-off-by: Sun Yuhan <[email protected]>

markpollack · 2025-05-30T14:50:45Z

Yes, we will review. Thanks

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaApi.java

…tionMetadata`, and added corresponding unit tests. Signed-off-by: Sun Yuhan <[email protected]>

Signed-off-by: Sun Yuhan <[email protected]>

TonyReggae · 2025-06-21T09:05:21Z

可以考虑加一个可自由扩展参数的字段，可以灵活应对兼容参数并避免延迟于ollama的更新。

sunyuhan1998 · 2025-06-24T01:47:24Z

Hi @markpollack , I hope you're doing well!

I just wanted to check in and see if there are any further changes needed for this PR. Let me know if there's anything else I should address — happy to make adjustments if needed!

Thanks again for your time and review!

hbsjz-swl · 2025-06-24T02:38:29Z

I would like to ask: If this PR is merged, will this feature be included in the snapshot version library? Then how can developers who are currently using the GA version use this feature?

sunyuhan1998 · 2025-06-24T02:59:43Z

I would like to ask: If this PR is merged, will this feature be included in the snapshot version library? Then how can developers who are currently using the GA version use this feature?

It seems to me that this feature should be included in 1.1.x, as well as being backport to 1.0.x. By the time the 1.0.x version is officially released, users currently using the 1.0.0 GA version will be able to use it by upgrading to the 1.0.x version

…-thinking # Conflicts: # models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaApi.java # models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaImage.java

markpollack · 2025-07-17T17:11:47Z

yes, I will backport it. Sorry about the delay. Only now getting my project mgmt hat on full time @sunyuhan1998

sunyuhan1998 · 2025-07-18T01:05:38Z

yes, I will backport it. Sorry about the delay. Only now getting my project mgmt hat on full time @sunyuhan1998

no problem at all! Please feel free to let me know if any changes are needed.

markpollack · 2025-07-29T05:57:13Z

i'm still working through this. exploring thinking more generally as an overall feature.

sunyuhan1998 · 2025-07-29T07:26:06Z

i'm still working through this. exploring thinking more generally as an overall feature.

No problem. If there's any part I can contribute to, please feel free to let me know. I'd be happy to get involved.

ilayaperumalg · 2025-10-21T12:08:44Z

@sunyuhan1998 Sorry about the delay in getting this in. Could you rebase this PR from the latest main please? Thanks

Signed-off-by: Sun Yuhan <[email protected]>

…tionMetadata`, and added corresponding unit tests. Signed-off-by: Sun Yuhan <[email protected]>

Signed-off-by: Sun Yuhan <[email protected]>

…to feat-support-ollama-thinking # Conflicts: # models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaApi.java # models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaApiHelper.java # models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaModel.java # models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java # models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaChatModelMetadataTests.java # models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaImage.java # models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/api/OllamaApiIT.java

Signed-off-by: Sun Yuhan <[email protected]>

sunyuhan1998 · 2025-10-22T01:27:09Z

@sunyuhan1998 Sorry about the delay in getting this in. Could you rebase this PR from the latest main please? Thanks

Hi, I've rebased this branch onto the latest main branch and properly resolved all conflicts.

I sincerely apologize for the messy commit history in my previous submissions. This was caused by an earlier merge of the main branch into the current branch, which led to some issues during the subsequent rebase process. However, everything has now been cleaned up, and the final code has been restored to a proper state.

Additionally, I'd like to clarify: the changes originally included in this PR to add think functionality support in OllamaApi have since been implemented and merged via #3969. As a result, after this rebase, the current PR retains only the changes related to Options and OllamaChatModel.

The PR has now been updated to its latest state and is ready for review. Thank you!

markpollack · 2025-10-30T19:36:24Z

Hi. Thank so much @sunyuhan1998 Regarding sending the previous thinking back, this comes up in other models as well now. We can perhaps stick it in the metadata of the message for now. The design of the messaging has changed, i think we can improve the model in 2.0, most leading models have 'thinking'.

markpollack · 2025-10-31T12:28:02Z

spring-ai-model/src/main/java/org/springframework/ai/chat/prompt/ChatOptions.java

+	 */
+	@Nullable
+	default Boolean getThink() {
+		return false;


We can't add 'think' to default chat option as a simple boolean is not sufficient to capture the full range of options across models, e.g. there is often a thinking token budget or other parameters.

Indeed, initially this attribute was designed solely to enable or disable Ollama's thinking. If we consider it to represent the entire content of "thinking" and intend to apply it across all models, it should be a complex object. I'm currently considering what properties we should include in it. Do you have any more specific suggestions?

The semantics of Ollama are: when not explicitly specified (null) → the server embeds the thought process within the tags in the content; explicitly false → thinking is disabled; explicitly true → separated into message.thinking.
The current ChatOptions interface provides a default implementation: default Boolean getThink() { return false; }. Although specific implementation classes have overridden this to return nullable values, if any implementation fails to override it in the future, "not set" will be treated as false, thereby altering the default behavior.

markpollack · 2025-10-31T12:30:54Z

...-ai-model/src/main/java/org/springframework/ai/model/tool/DefaultToolCallingChatOptions.java

+	@Override
+	@Nullable
+	public Boolean getThink() {
+		return this.think;


As with ChatOptions, the boolean itself isn't sufficient across all model providers.

markpollack · 2025-10-31T12:51:48Z

models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/api/OllamaApiIT.java

 		assertThat(response.done()).isTrue();
 		assertThat(response.message().role()).isEqualTo(Role.ASSISTANT);
 		assertThat(response.message().content()).contains("Sofia");
 		assertThat(response.message().thinking()).isNotEmpty();


this assertion isn't passing.

I'm using the Qwen model, and it works fine on my side. Perhaps we need to adjust this assertion test, as the model's output has some randomness. I'll modify it soon.

markpollack · 2025-10-31T12:52:19Z

models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/api/OllamaApiIT.java

+		assertThat(responses.stream()
+			.filter(r -> r.message() != null)
+			.map(r -> r.message().thinking())
+			.collect(Collectors.joining(System.lineSeparator()))).contains("Sofia");


this assertion is failing for me.

markpollack · 2025-10-31T12:52:37Z

models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/api/OllamaApiIT.java

+		assertThat(responses.stream()
+			.filter(r -> r.message() != null)
+			.map(r -> r.message().thinking())
+			.collect(Collectors.joining(System.lineSeparator()))).contains("solar");


this assertion is failing for me.

markpollack · 2025-10-31T12:53:47Z

Moving to 1.1 RC1 due to the need to move 'think' out of the high level chat options interfaces and the debug the test failures.

shuanggegege · 2025-11-03T09:57:33Z

、、

Moving to 1.1 RC1 due to the need to move 'think' out of the high level chat options interfaces and the debug the test failures.

Hello, may I ask if this feature will no longer exist in version 1.1.0-M4

ilayaperumalg · 2025-11-03T13:36:55Z

Hi @shuanggegege , yes, this feature is targeted to get merged in 1.1.0-RC1 hopefully.

shuanggegege · 2025-11-04T01:12:10Z

OK,I understand, thank you @ilayaperumalg

liugddx · 2025-11-04T15:02:35Z

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/OllamaChatModel.java

 				if (ollamaResponse.promptEvalCount() != null && ollamaResponse.evalCount() != null) {
 					generationMetadata = ChatGenerationMetadata.builder()
 						.finishReason(ollamaResponse.doneReason())
+						.metadata("thinking", ollamaResponse.message().thinking())


Currently, metadata is only constructed when both promptEvalCount and evalCount exist, along with "thinking". This results in some responses containing thinking not being recorded due to missing count fields.

Placing "thinking" directly as a top-level key in ChatGenerationMetadata may conflict with future keys of the same name from other providers. It is recommended to consider using a namespace prefix (e.g., "ollama.thinking") or defining the key name in a public constant to avoid collisions.

liugddx · 2025-11-04T15:04:41Z

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/OllamaChatModel.java

 			.messages(ollamaMessages)
-			.options(requestOptions);
+			.options(requestOptions)
+			.think(requestOptions.getThink());


Enter null, ensure it is not serialized into a JSON field to trigger the server's default behavior. It is recommended to set it only when non-empty think

Boolean think = requestOptions.getThink(); if (think != null) requestBuilder.think(think);

liugddx · 2025-11-04T15:05:55Z

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaModel.java

+	 * Qwen3 1.7b
+	 */
+	QWEN_3_1_7_B("qwen3:1.7b"),
+
+	/**
+	 * Qwen3 0.6b
+	 */
+	QWEN_3_06B("qwen3:0.6b"),
+


Among the existing constants, there is QWEN3_4B, while the newly added ones are QWEN_3_1_7_B and QWEN_3_06B. The naming styles are inconsistent (such as the use of underscores and the separation of numbers and units). It is recommended to adopt a unified style (for example, using QWEN3_1_7B / QWEN3_0_6B consistently or applying a uniform underscore scheme) to ensure readability and searchability.

liugddx · 2025-11-04T15:08:26Z

spring-ai-model/src/main/java/org/springframework/ai/chat/prompt/ChatOptions.java

+	 */
+	@Nullable
+	default Boolean getThink() {
+		return false;


The semantics of Ollama are: when not explicitly specified (null) → the server embeds the thought process within the tags in the content; explicitly false → thinking is disabled; explicitly true → separated into message.thinking.
The current ChatOptions interface provides a default implementation: default Boolean getThink() { return false; }. Although specific implementation classes have overridden this to return nullable values, if any implementation fails to override it in the future, "not set" will be treated as false, thereby altering the default behavior.

sunyuhan1998 added 3 commits May 30, 2025 11:24

feat: Remove problematic tweaks

af9958b

Signed-off-by: Sun Yuhan <[email protected]>

fix: Adjust the type of the "think" field in Ollama to support its de…

91fb151

…fault behavior, thereby ensuring compatibility with older versions of Ollama calls. Signed-off-by: Sun Yuhan <[email protected]>

sunyuhan1998 mentioned this pull request May 30, 2025

Accessing the New "Thought" Field from Ollama in the Spring AI Framework #3383

Open

feat: Add unit tests for ollama's think support. upgrade the test-con…

f77e08a

…tainer image version of ollama to 0.9.0 Signed-off-by: Sun Yuhan <[email protected]>

markpollack self-assigned this May 30, 2025

markpollack added the ollama label May 30, 2025

markpollack added this to the 1.1.x milestone May 30, 2025

markpollack added the for: backport-to-1.0.x label May 30, 2025

sunyuhan1998 mentioned this pull request Jun 10, 2025

Disable thinking mode for models used through Ollama like qwen3:0.6b #3497

Closed

markpollack reviewed Jun 11, 2025

View reviewed changes

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java Show resolved Hide resolved

markpollack reviewed Jun 11, 2025

View reviewed changes

models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaApi.java Outdated Show resolved Hide resolved

feat: Propagate the thinking returned by Ollama back to `ChatGenera…

b188adc

…tionMetadata`, and added corresponding unit tests. Signed-off-by: Sun Yuhan <[email protected]>

sunyuhan1998 mentioned this pull request Jun 16, 2025

Ollama thinking mode #3543

Open

feat: Add support for the think flag at the ChatMode level.

552a346

Signed-off-by: Sun Yuhan <[email protected]>

Merge remote-tracking branch 'upstream/main' into feat-support-ollama…

557a98e

…-thinking # Conflicts: # models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaApi.java # models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaImage.java

sunyuhan1998 mentioned this pull request Jul 3, 2025

Hope to provide support for Ollama's thinking mode #3738

Open

sunyuhan1998 mentioned this pull request Aug 4, 2025

Update Ollama API to latest version #3969

Closed

This was referenced Aug 26, 2025

How can I get the thinking content using a thinking model? #3954

Open

Add 'think' option to control reasoning models' thinking behavior #4233

Closed

ilayaperumalg removed the for: backport-to-1.0.x label Oct 16, 2025

ilayaperumalg added this to the 1.1.0.M4 milestone Oct 16, 2025

sunyuhan1998 and others added 4 commits October 21, 2025 20:41

rebase: rebase from the latest main

afc3ba2

Signed-off-by: Sun Yuhan <[email protected]>

rebase: rebase this PR from the latest main

3411fe2

Signed-off-by: Sun Yuhan <[email protected]>

feat: Propagate the thinking returned by Ollama back to `ChatGenera…

abfc9e1

…tionMetadata`, and added corresponding unit tests. Signed-off-by: Sun Yuhan <[email protected]>

feat: Add support for the think flag at the ChatMode level.

f61a99f

Signed-off-by: Sun Yuhan <[email protected]>

sunyuhan1998 closed this Oct 21, 2025

sunyuhan1998 force-pushed the feat-support-ollama-thinking branch from 379ea2c to a897177 Compare October 21, 2025 12:57

sunyuhan1998 reopened this Oct 21, 2025

sunyuhan1998 added 5 commits October 21, 2025 21:08

fix: format code

2f9417a

Signed-off-by: Sun Yuhan <[email protected]>

fix: format code

dfb2522

Signed-off-by: Sun Yuhan <[email protected]>

fix: fix ut

64bee7b

Signed-off-by: Sun Yuhan <[email protected]>

refactor: rename isThink() to getThink()

faa8b7c

Signed-off-by: Sun Yuhan <[email protected]>

markpollack reviewed Oct 31, 2025

View reviewed changes

markpollack modified the milestones: 1.1.0.M4, 1.1.0.RC1 Oct 31, 2025

guanxuc mentioned this pull request Nov 3, 2025

We need the think parameter of ollama to allow users to decide whether to turn on or off think. Currently, we cannot control the model to stop thinking #4781

Open

liugddx reviewed Nov 4, 2025

View reviewed changes

Added support for the "think" for Ollama #3386

Are you sure you want to change the base?

Added support for the "think" for Ollama #3386

Conversation

sunyuhan1998 commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sunyuhan1998 commented May 30, 2025

Uh oh!

markpollack commented May 30, 2025

Uh oh!

Uh oh!

Uh oh!

TonyReggae commented Jun 21, 2025

Uh oh!

sunyuhan1998 commented Jun 24, 2025

Uh oh!

hbsjz-swl commented Jun 24, 2025

Uh oh!

sunyuhan1998 commented Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markpollack commented Jul 17, 2025

Uh oh!

sunyuhan1998 commented Jul 18, 2025

Uh oh!

markpollack commented Jul 29, 2025

Uh oh!

sunyuhan1998 commented Jul 29, 2025

Uh oh!

ilayaperumalg commented Oct 21, 2025

Uh oh!

sunyuhan1998 commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markpollack commented Oct 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markpollack commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shuanggegege commented Nov 3, 2025

Uh oh!

ilayaperumalg commented Nov 3, 2025

Uh oh!

shuanggegege commented Nov 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

sunyuhan1998 commented May 30, 2025 •

edited

Loading

sunyuhan1998 commented Jun 24, 2025 •

edited

Loading

sunyuhan1998 commented Oct 22, 2025 •

edited

Loading

markpollack commented Oct 31, 2025 •

edited

Loading